>>623381A medium will fit torso 18-21 inches and a large fits 20-23 inches.
Your torso measurement is the distance between the top of you illiac crest (hip bone) to your c7 vertebra (the one that makes a bump when you put your chin down.
As a former packfitter, I had a lot of guys convinced they were larges because they were tall, fat or just thought it was more manly to buy a large. But it's all about proportion, not height.
Most guys are in the medium size range. If you're having a hard time measuring yourself, a decent specialty outdoor shop like Cabela's or REI will be able to help you.