There's nothing else like Monster.
A good thriller is
Perfect Blue. It benefits from Satoshi Kon's great directing skills.
Not a thriller, but a good detective story where people are people and are driven by believable reasons is
Kaze No Youjinbou. The pacing is slow but just about right for the story. Kind of like the pacing in Monster fit it perfectly.
For realistic characters, but once again not a thriller, there's the
Master Keaton series. And you may add
Yuugo Koushoujin. These two focus on conflict negotiators. The characters are mature people, and they use realistic means to achieve their objectives. The stories in Master Keaton are very human in nature, just like those in Monster were. The character designs are also similar to those in Monster. There are no supernatural forces or other things out of this world and that don't make sense.