I know it was said somewhere that Voban never learned magic not because he sucked but because of preference or something. It was said in the anime, too. It's the same with Doni, he won't learn magic because he's a sword freak. You might think learning magic would be useful and logically that's true, but these guys wouldn't be Campione's if they relied on magic.
Voban didn't learn Japanese quickly just because of the large amount of magic he has.