look I agree that rape is bad and nobody should have violence done to them, but isn't a big part of the "damage" done totally subjective societal shame that comes from us making such a big deal of it?

If you were raped and lacked the cultural knowledge to understand that it was such an egregious and shameful thing, you would be relatively unharmed and get over it pretty quick.