For what it's worth, I don't think you are being a coward, You just feel conflicted, and that is fine. Feel free to ignore this, but here is my take on it:
If I were in your situation, I would treat him as a familar face that you haven't seen in a while. What I mean is, keep your interactions with him brief and superficial. And keep it civil. Admittedly, ignoring your own concerns isn't good, but this event isn't the time to rehash any drama that may still be there. After all, the guy and your mom are still on good terms, so he can't be a total ass, right? Then again I don't have a good relationship with any of my exes, so maybe you should just do the complete opposite...