Note that I'm a hardcore RPG collector... I own almost every non-action RPG in both the NDS and the PSP, and what I'm going to say is based upon my bias towards RPGs.
First off, lets go hardware. Definitely PSP is better in terms of graphics, and this isn't more obvious in any other game than Puzzle Quest (which I recommend btw, because despite its seeming simpleness, it's highly addictive). Battery life is on the NDS though, as the lite version just keeps going... and going... and going... I can use it up to three days at a time without charging, while the PSP I have to charge every day just so it doesn't die on me in the middle of a battle (most RPGs won't 'save state' in a lot of particular events, like battles for example).
From a games standpoint, I find the NDS to be far superior. Putting aside that it can play the GBA library, NDS still has better games. Better as in original... what does the PSP have? Ports after ports of RPGs. And of those ports whose originals haven't been in English yet, most suffer from major lag problems (NISAmerica games are particularly guilty of this). So much so that I went through the trouble of using custom firmware and ripping my bought UMDs to the stick just to decrease lag and load times.
And how about price? While the PSP is reasonable now, getting a decent memory stick will still kill you, especially if you want the biggest they have to offer. I got my 8 gig stick for as much as a PSP when the stick first came out, and my 2 gig stick for half a PSP when I first got my PSP. So already, I bought 1.5 PSPs just on sticks. Though to be fair, you can live without the biggest sticks, so you can skimp if you decide not to.
Customer service goes towards the NDS as well. If you're a perfectionist, if there's a dead pixel in your NDS, Nintendo will replace it no questions asked. My first NDS was repaired (or replaced? I forgot) within 2 weeks, and I spent at most 10 mins in the phone. Sony? Pfft... you're likely to jump through many hoops like I did with my first PSP. At a certain point, I figured out that the time I was wasting through their customer service would've been better spent earning money towards a new PSP, so I gave up.
My verdict? NDS hands down. Better bang for your buck, definitely, so if you're an RPGamer on a tight budget, go get the NDS. Not that I don't think a PSP purchase is not at all unreasonable; you get Jeanne D'arc, IMO better than anything the PSP or the NDS (or even the PS3!) has to offer, as its story and system are reminiscent of Tactics Ogre. Considering I'm a very big Tactics Ogre fan, I don't think I can give a game better recognition than that. You also have some ports that's great if you missed them the first time around, like Final Fantasy Tactics and Valkyrie Profile (just to let you know, getting the original VP in very good condition would probably cost you as much as a PSP lol).
But I LOVE my NDS. Remember the NDS I got repaired? It's already broken BECAUSE I used it too much even only after 6 months of use (I half beheaded the top half from the bottom half
), and I had to get another one. Well worth the money though, and even then, I still spent less than I did for a PSP alone. GBA games like Yggdra Union and Super Robot Taisen, then you have NDS games like Phoenix Wright, Luminous Arc, Etrian Oddysey, etc. and now Front Mission, with FFXII:Revenant Wings coming in near future and Dragon Quest 9 coming in the not so near future for games. I really don't think an RPGamer can go wrong with the NDS.
Of course, there's the homebrew scene, in which if you decide to spend time on, you can have your PSP do things like emulate SNES and PS1 games. I myself haven't tried it despite custom firmwaring my PSP, since of course, I probably own those games and have beaten them already lol. Be aware though that homebrewing would probably need a big stick, so be prepared to spend some more dough.