Hiro Shimono isn't the problem. The only way he could be is if his voice was so terribly unfitting (e.g.
this and
this) that it becomes unintentionally funny or annoying. Shimono fits Keima's character well enough.
One problem is with the tone of Kanon and Shiori's arcs. The anime begins with a close adaptation to the manga and begins their arc on a light, humorous note. Then the next episode it's completely serious. The transition is fairly abrupt and inappropriate. If the anime wants to keep the comedy, then it needs to be kept throughout the entire arc. If the anime wants to make it a serious romance, then the comedy should have been toned down from the start, even if it is unfaithful to the manga.
The second problem is the girls (or lack thereof) this season. The story doesn't pick up until Haqua and Tenri appear, and I'm sure plenty of fans are partial to at least one of the girls past Shiori. If the studio had made up its mind on whether to make the anime a comedic parody like the manga, or a serious romance like they did for the latter half of Kanon and Shiori's captures, the anime would have been better received as a seious romance, or fit more girls in if they decide to stick to the comedy.