Would a size medium be too small for an average-build man? By AFLGO. Posted on April 18, 2019 A size medium will fit snugly on an average-build man. Men who are “5’8” and taller who weight 170 pounds or more should wear a size large or larger to get the best fit.