>I fucking laugh and kek at gymcel ogres thinking they can do both
There is still somewhat of a correlation I think between being scary to men and being attractive to women. Being tall or having a stronger jaw for example, both make you more intimidating, and they are also traits that women find desireable. Of course it is not a direct correlation, obviously the man most intimidating to other men is not going to be the most attractive man in the world, but there certainly is a fair amount of overlap.
To answer your question, being attractive to women is more important, since you gain little benefit to being intimidating to other men because we live in modern society where if you fight someone you get sent to prison for assault. Being intimidating to men is only really helpful if we were in the wilderness, or maybe if your occupation relies on it like if you are a TV wrestler or something.