There is
absolutely no disadvantage to using non-mod16 resolutions except that it means you need a tiny bit more bitrate to store the extra rows. As an example, if you're encoding 800x596, that will take the same amount of bitrate as 800x600. As you can see, the difference is absolutely negligable--and nobody cares.
The way x264 handles mod16 is by "mirroring" the rows of pixels over the edge of the frame to fill the mod16 boundary. This avoids introducing compression artifacts and such that other methods prior sometimes did.
If a hardware player "didn't support non-mod16", it couldn't play 1080p
, because 1920x1080 isn't mod16 either.
"Supporting non-mod16" actually means "supporting post-decoding cropping," because all streams as encoded at mod16 no matter what because of the mirroring/padding I described earlier.